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© Transform coding using coefficient prediction techniques. 



© A system and method for improving the quality 
of decoded images, without any cost to compression 
efficiency, in both progressive still frame and motion 
video applications wherein the images are coded 
with intraframe transform based coding techniques, 
such as Discrete Cosine Transform (OCT) coding. 
Prediction of the AC components is utilized and 
operations are performed on the AC coefficient pre- 
diction in the decoder portions of the system in 
combination with a thresholding technique. Input im- 
ages are subdivided into blocks of pixels, which are 
changed into blocks of coefficients by forward trans- 
form coding resulting in DC coefficients, representa- 
tive of the average value in a block of pixels, and AC 
coefficients, representative of harmonic frequencies 
in a block of pixels. AC coefficient prediction is 
performed using both DC and AC coefficients from 
the block, whereupon the predicted coefficient val- 
ues are thresholded and the thresholded coefficient 
values are added to their corresponding transform 
coefficient values for the given block. These steps 
are repeated for ail of the blocks of the image and 
the image representation is recovered by applying 
an inverse transform operation to the results of all of 
the additions for all of the blocks of the image. 
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Transform coding using coefficient prediction techniques 



Description of EP0422404 



TRANSFORM CODING USING COEFFICIENT PREDICTION TECHNIQUES 

The present invention relates to electronic image handling and more particularly to a ^ st « m ^ ^ thod 
fnr nnpratina on imaoe data that has been transform coded, e.g., motion video data, using coefficient 
^SSS^SS^^SS^ ^over the coded image while facilitating efficient compression of the 



video data. 



Efficient motion video compression systems typically use a combination of intraframe and ^ertwme 
SteSesIn intraframe coding a video frame (or field) of a sequence is coded without reference 
to 2 £me (or Sd) in the video sequence. In interframe coding a video frame is coded by 
Sferenc no one adiS previously coded frame. Since the typical information content of consecutive 
S£ fftSSTSSS very fwe compression of frame differences, and thus interframe coding, ,s typ.caHy 
more SSStlSSSSle coding. P Efficiency can be further * 
frame motion compensation techniques to the bas<c interframe coding, as , de ^nbed fo nnstence by H. 
MUSSMANN P PIRSCH. and H. GRALLERT, in "Advances in Picture Coding Proc. IEEE, 73, 523 
iSsSH fh > thus desirableto use interframe coding as much as possible to optimize compress,on rate ,n 
video sequences, but, nevertheless, intraframe coding cannot be avoided. 

Intraframe coding is particularly necessary to handle scene changes and also to meet certain proposed 
eSemente arising from recent standards efforts for motion video compression. Regard.ng hese 
Sards r^em'ents, it appears that if certain proposals are adopted it will be 

access anv video frame in a sequence within certain time limits, and to be able to playback a prerecoraea 
^UJueJShSS forward and fast reverse by accessing and skipping frames wrth a certa.n 
"SI-! q-Tch renu rements call for at least the periodic use of intraframe cod.ng. For example, a fast 
Krd mode at ToZes fhe norlS speed would require accessing, decompressing, and dis P lay.ng a 
vid^o frame ■ everv 10 frames in the sequence. Since interframe coded frames need to refer to adjacent, 
nSS, lv demdk ^frames they cannot be used to implement this function.Only intraframe coded frames 
^SS^SiMtSSS^^LB and thus a fast forward function at 10 times the norma, speed 
^nutL an int™me coded frame for every 10 frames. It follows that the length of time required to 
^SSXt!iS^fS^nL video sequence will be constrained by the maximum number of frames 
between ^intraframe break? Unfortunately, the frequent recurrence of intraframe breaks degrades the 

Darticular can sSw very annoying blocking artifacts at low bit rates. These artifacts are particularly 
notiS^ smooth areas of the image and they appear as a periodic flashing of blocks wh.ch the 
interframe coding then gradually tends to correct. 

Referrina to the commonly-applied transform coding technique, it can be adapted for use with either the 
mSSL o interframe mode In transform coding, particularly Discrete Cosine Transf °rm Coding (DCT), 
anTmSSe i ^typ Subdivided into blocks of, e.g., 8x8 pixels, each of which is then transformed through 
ma hemat cal operations into a new representation of the original block of pixels. The "pixels" . .the 
^ansfoTmed i blocks are called transform components or coefficients Known examples <* tansrfonm 
oneraSons are the DCT Haar, Hadamard, and lapped overlap transforms. In all cases the transform 
ope a ion ha ^the property of compacting the energy of the original pixel blocks into a few transfonr, 
romDonentrsuch that only those coefficients need to be quantized, compressed, and transm.ttedWhen 

«mpoSeri.a re quantized coarsely, which is the case with high compression rates typical of 
motion video standards, this block structure introduces "blocking" artifacts. 

A tPrhniaue is described by B. NISS in "Prediction of AC coefficients from the DC values", 
^0?S 7 /SC2WG8 N74o! May 1988. and "Prediction of AC coefficients from the DC values revisited . 
JPEG 147 ^ ISOrrC97/SC2AWG8 document. June 1988. for improving the quality of a stil frame that is 

Sled DC coeffTcient signal DC and a number of AC-coefficient signals AC. The DC coefficent .s 
rSrese^ average value in the original block of pixels, while the AC-coeffic,ents represent 
lar iouTharmonic frequencies therein. Initially, only the DC components are quanfzed Q in a quanfrzer 20 
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transmitted to obtain the coded image. 
DC component. 

S wSd not otKse be possible at low-bit compression rates.Additionally, and most significantly, .t .s 
desiraSe that th ^^stmprovement be achieved with no penalty in compression efficiency. The -nvenbon 
solves he , DroSem ofS^rting "blocking" artifacts that result with intraframe coding while maintaining 
frnage i Sri* by appropriately uti.izing transform cod-ng techniques. 

tect.KS by using prediction of the AC components. The invention involves operating, orr the AC 
Sufficient ^ P rediction P which operation takes place only on.the decoder portions of the system .n 
combination with a thresholding technique. 

The svstems to which the invention is directed essentially operate by subdividing input images into blocks 
rrf ni^Zi? These Wocks of pixels are changed into blocks of coefficients by a forward transform cod ng 
ope atio^ indude: DC coefficients, representative of the average value in a 

Woe Tof p"ixe"s- and AC coefficients, representative of harmonic frequencies .n a block of pixels . The 

of the additions for all of the blocks of the image. 

Tvoicallv the transform coefficients for the blocks will be quantized so that the predicted coefficient values 
IZ Quantized before ErSholding, and the results of the additions are dequantized before applying the 

^SSSS^SS^^b addition step, whereupon compensation for the scaling .s performed after 
applying the inverse transform operation. 

Decoder embodiments for both progressive still frame and motion video applications are disc |°s^. 
Rg X 1 % ^oSSagram illustrating a progressive still frame OCT encoder ,n accordance w.th the pnor 

Rg. 1b is a block diagram as in Fig. 1a illustrating an alternative progressive still frame DCT encoder in 
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%^^S^SL*ng a OCT decoder for use in combination with the OCT encoder shown in 

p!o \% a block diaaram illustrating a typical DCT motion video encoder in accordance with the prior art. 
Pin 4 s I block diaaram us rating a typical DCT motion video decoder in accordance w,th the prior art. 
Fig. 5a *s ,1 S S^KSX a DCT motion video decoder in accordance with the present 

FkTstotea block diagram illustrating an alternative embodiment of a DCT decoder in accordance with the 
presennnverlSon sk Fi£ 5c is a block diagram illustrating yet another alternative embod.ment of a DCT 

F ° C t^^ stiH frame encoder in accordance with the present 

Rg^bTs a block diagram illustrating a DCT progressive still frame decoder in accordance with the present 

Sg^Ta diagrammatic representation in illustration of scaling factors used in connection with the 

Fig 6 ^ flowchart illustrating the computation of AC predicted values using a thresholder according to 

Fig. gTaSowchart illustrating the computation of two threshold values of the thresholder of Fig. 8 used in 
AC prediction. 

The oresent invention involves a system and method for improving the intraframe coding of electronic 
Kgw^uSIJvStous embodiments respectively capable of treating data represent^ either mot.on 
video sequences or still frames. 

Firstlv with regard to the motion video implementation, reference is made to Figs. 3 and 4 which show 
Knonfied diagrams of a typical prior art motion video encoder and decoder, respectively using, e.g., the 
DCT a^ough X ^ransformscould be used without .oss of generality. The upper path of the ^encoder of 
Fig 3 incSng transformer 10 and quantizer 20. is the encoder proper and w.ll be referred to as the 
"transmission encoder". 

For intraframe coding switch S1 in Fig. 3 is in position 1 so that the input image data to the transmission 
encoder i^ •dTeSy operated upon by the DCT transformer 10, quantized in quantizer 20 and then 
fransml ted It noted S the quantized outputs will be typically entropy coded in a coder 50, us.ng 
Huffman or'ari hmetic coding techniques, before transmission. However, such addrt.onal operations or 
aoomoriate exteSs w Ibe chosen b^. and within the purview of. those of skill in the art and need not 
SSSSSSSSSS?^. in the decoded in Fig. 4. switch S2 is also in position < ^^^^ 
sianal to be deouantized (Q-<1>) in dequantizer 25. and inversely transformed (IDCT) in inverse 
Sslormer i^befo e being directly output as the reconstructed image, as well as being stored .n a 
SS^rtion frame bufferGO.The output of buffer 60 may be summed with the output of .nverse 
transformer 11 in a summer 43. 

The encoder of Fig 3 also contains a decoder 70, such as that of Fig. 4. in its feedback loop so that the 
SinstScted lage can be used as a prediction for the next frame by subtracbng •[^££" n 
there may be appropriate extensions to the basic coder/decoder (codec) of F gs. 3 and 4, such as he 
ntrodSn of motion compensation and filtering of the predicted frame For interframe cod.ng sw.tch SI 
s in dos i ion 2 so that frame difference signals from summer 42. rather than original I frame signals on l.ne 
?4 iS^stoSed wd quantized by the DCT transformer 10 and the Q quantizer 20, respect.vely 
LSrrwitch 2 of the decoder is in position 2 so that frame reconstruct™ ,s obta.ned by add.ng the 
reconstructed frame differences from summer 43 to the prediction (prev.ous) frame. 

As previously noted, at high compression rates, the DCT AC-components are quantized so coarsely ^that 
me? ?re simply not detected in smooth areas of an image, thus introducing "block.ness ^ Jhew artfecte 
are typTcl of block-oriented transform techniques and they cannot be easily corrected other than by the 
use of finer quantizers which increase the transmission bit rate or bandw.dth requirements. 

Prediction of AC-coeff.cients is one alternative to be used to correct block artifacts. The scheme shown in 
F q e cannot be used, however, because it assumes that the AC predictions are quant.zed 0 ! wrth , the 
sSe quaSzer 20 as the original AC coefficients. Since this same quantizer was too coarse to avoid the 
Sets in me first place, it does not help with the prediction either. It is possible .to move the AC quantizer 
?21 in Fio a) n the system to the position of quantizer 22 shown in Figure 1 b; but, in such case large 
predion erro s may occur in active/complex Ireas of the image thus degrading compression effiaencyJn 
e ther case he complexity of a motion video encoder incorporating the arrangements in F g la or Rg lb 
fnpSce oi ?the "transmission encoder" of Fig. 3, would be increased sigmficantly. as the pred.ct.on c.rcu.try 
would have to be replicated in the feedback (decoder) path. 
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Consequently for intraframe coding in accordance with the present invention, the encoder may be 
esseS of he form of the "transmission encoder" shown in Fig. 3. However, the feedback decoder 70 
3f be o f a term such as shown in either Fig. 5a. Fig. 5b. or Fig. 5c. It will be seen therefore that any 
^crease in complexity in the system is limited to the decoder and accord.ngly in the encoder to the 
decoder pat^ Further, as will be seen below, there will be no degradation in compress.on efficency. 

Turning to the decoder embodiments shown in Figs. 5a. 5b, and 5c, all will be noted to incorporate the 
coSaL of 1) performing an AC prediction on both the DC and AC coefficient 2) a 

thresholdino operation P. More particularly, in the preferred embodiment of Fig. 5a, the DC and AC 
S^SSSbStore scale up at 75, are fed over line 91 to AC predictor 32. This predictor may be 
Sv imp lemen ted by a combination of simple arithmetic operations, such as that disclosed in the 
M^CHELL ET AL paper cited above and the use of lookup tables. The prediction coefficients output by 
oredictor 32 are quantized q in a fine quantizer 24. that is, one that produces a quantization q which is 
finer San the quanfeation Q of quantizer 20 in Fig. 3. The finely quantized coefficients from fine quantizer 
STlKlSH^ P in a suitable thresholder SO.Thresholder 80 may ^^^jj^ 
table or a comparator, as appropriate for the operating parameters. The thresholded AC coefficients 
output SoWer 80 are summed with their corresponding scaled "P AC coefficient inputs ; on , line ,89 
°n a summer 44 When switch S3 is in position 1, the output of summer 44 w.l be sent to the remainder of 
L decker wh ch essentially constitutes the components of the decoder of Fig. 4 w, h the ^addition of a 
scale down function at 76. to compensate for the earlier scale up. The scale up function on the AC 
coefficients n 75 does no necessarily add complexity to the system as there may be a sea e up function 
£ * ^quantizer 25. The scale down function 76 may also be incorporated in the inverse transformer 1 1 . 
buUn any event, appropriate scaling in these combinations will be within the purview of those of skill in 
the art. 

The first important feature to be noted in the inventive system, which has implications on implementation 
complexity is that AC prediction is done only in the decoder portions. Thus, unlike the prior art. with the 
inveKfon iHs nSnecessary to transmit the prediction difference, but rather the original AC coefficients 
nemselves are transmitted Prediction is only used to improve the quality of the decoded image; .t has no 
5SSoSi?JoSS rate. It has been determined experimentally that with prior art techniques such as 
hat of NISS. AC prediction has a greater impact on image quality than on compress.on efficiency Jhe 
second important feature is that unlike the decoder shown in Figure 2. a finer quantization, ^ q is used for 
the predion coefficients. In this manner it is possible to predict AC values wh.ch coukl not b ^efetfed 
by the coarse quantization Q. In problem areas, such as flat areas of the image, the AC coefficient i tend 
to be small so that coarse quantization Q will miss them. Thirdly, in accordance with the invention, after 
maD D fnq th! AC predictions through a thresholder operation, P, the predictions are added to a scaled 
So! the To^gS quantized AC coefficients. It is preferred that q be a linear scaled down version of Q. 
e.g., q scale down = DC, AC scale up. 

With regard to scaling, attention is called to Fig. 7 which diagrams quantization scaling factor* ^hereto 
for instance a scaling factor of four is shown. It will be seen that Q1 = q4 and q1 = QJ4. This same scaling 
factor wSS be applied to scale up the DC. AC components in Fig 5a. Another .mportant ; aspec t of the 
nvention is the use of thresholder P. Without proper thresholding large errors could be hn the 

AC prediction estimates. The thresholding function P of the invention is used to insure that onfy AC 
oredictions which could not be detected and quantized Q by the coarse quantizers are allowed, i.e.. 
p red £ ted valeTwNch can be detected by the coarse quantizers are simply set to zero Thus, assuming 
KS^nSr decision levels are in between the reconstruction levels. an< that ^nything; Om 
would be detected by quantizer Q. the mapping generated by P forces q1 = 0 for ql >/= Q1/2. _Th,s 
mapping implicitly allows the generation of AC-prediction for smooth areas of the image only_The .AC 
predictions for active areas - where predictions would normally be large - are forced to zero .The smooth 
areas are thus substantially improved in quality. Since the decoder can calculate the prediction from 
known^atotte improvement in quality comes at NO to 
a far superior image quality is obtained when the image is reconstructed in this manner as compared to 
the use of no AC-prediction. 

Another decoder embodiment in accordance with the invention is shown in Fig. 5b. ^erein scale .up > of the 
DC and AC coefficient inputs is performed at 75 min prior to their being fed over respective fines 92 and 
9?to AC rSedSES mto . The output of predictor 32 min is quantized Q in quantizer 24 m.n and 
toreshoidS Tin toreshoSer 80 min before being fed to summer 44. With this arrangement the scale down 
function ov T the AC prediction quantizer 24 of Fig. 5a is not needed in quantizer 24 mm . however, greater 
Setir P reciSon P and larger lookup tables are required for implementing the prediction function in 32 
min . In this case, the mapping generated by P forces q1 = 0 for q1 >/- 2Q1. 

A further implementation of a decoder in accordance with the invention is shown in Fig. 5c. In this 
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*™KoHim^nt inverse Quantization Q<-><1> is performed by inverse quantizer 26 on the DC and AC 

and multiplies may be preferred. 

While the invention has in the preceding embodiments been described as applied to processing motion 
Seo data The i nventive principles disclosed can also be applied to the progressive transmission oU st.ll- 

encoder of Fig. 1a. 

In the Fio 6a embodiment the portion AC min of the AC-coefficients AC constitute those portions of the 
coefficient,^ ^fhTare "coarsely quantized". For example, in an 8-bit sign-magnitude representation of 

the thresholder forces the prediction to be zero (no predictions. 

Secondly if AC min =0 the thresholder will allow quantized predicted values less than 63; larger values 
are set to ze?o This threshold is chosen because a larger AC value would have been detected by the 
value of AC min in the first place. 

Pnr ap min -0 -in successive stages of progression- the difference between the original quantized AC 
valueJa^d their predS Since no sign information was transmitted wrth AC : min the 

sfgn can be transSd as soon as the first non-zero magnitude bit of the difference ,s sent. For AC mm 
NOTEQUAL 0, the remaining least significant bits (LSB) are simply transmitted. 

82 Sire being summed with DELTA AC signals in summer 46. The operation ol Ihis decoder reverses 

WormE tSZ ml i SiTh e prediction is threshoided before it gets added to the dtfferenee data 
transmitted in later stages of the progression. 

it r^ht notPd that in the event that the quantization is not linear in the cited embodiments, the 

earned out by defining the threshold values differently. 

Bv wav of example flowcharts are shown in Figs. 8 and 9 of software implementations of N'SS'sAC 
of the form of that in Fig. 7. 

AC PREDICT In Fio. 8 five AC coefficients are predicted. COMPUTE_THR calculates THR1 and THR2 
fo?iac?Scient. For each coefficient a quantized DC gradient is ca.cu.ated accordmg to the formulas 
taken from NISS and MITCHELL ET AL. 
GRAD(0,1) = (DC4-DC6) 
GRAD(1,0) = (DC2-DC8) 
GRAD(2,0) = (DC2 + DC8 - 2 x DC5) 
GRAD( 1 ,1 ) = (DC1 + DC9 - DC3 - DC7) 
GRAD(0,2) = (DC4 + DC6 - 2 x DC5) 

^THK^^GRAD^ > -THR2(i,j) the prediction is -1 . Note that the order of the questions is important 
http : //v3.espacenet.com/textdes?DB-EPODOC&IDX-EP0422404&F=0&QPN=EP0422... 6/16/05 
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sinre it is possible that THR1 and THR2 are equal. Also note that the gradients are obtained by using 
quantized Dc SlVs. Finally note that the resulting predictions are stored in the same DCT array where 
AC_DECODE adds its decoded values. 

COMPUTE THR- Fig. 9 illustrates the computing of the thresholds used in AC prediction A set of two 
ttresholds are computed for each AC coefficient predicted. These thresholds incorporate the 
Hpnuan ization of the DC gradient, Q0.09; and the reduction of the AC quantizer step, Qi,j/4 (see F g 7). 
S^SSSl7a«d2S^d from N.SS after a scale up of a factor of 2<1 ><2>, so as to P^Unteger 
Sthmetfc For example, K(0,1 )=9.1 1 08x4096. These formulas require further normahzation by 2<6> -64 
so that the total normalization becomes 2<6> 2<1><2> = 2<1><8>. The .nteger values of K are. 



K(0,1) = 37318 
K(1,0) = 37318 
K(2,0) = 9136 
K(1,1) = 5313 
K(0,2) = 9136 



nPi ta renresents the unnormalized contribution of a step of one in the DC gradient to the quantized AC 
SSffid^THWco^^S^mber of DC gradients steps needed to generate a quantized AC prediction 
of vSue 1 THR2 counts ,Z 'number needed to generate a quantized AC prediction of value 2. The .mt,al 
value of TEM=2<1><7> is used for rounding. 

it will accordinolv be seen that a system and method are disclosed for improving image quality in 

ntra raSX™ baled coding techniques by using prediction of the AC components, wh.ch pred.ct.on 

lakes pEcJ I only^The d^oder portions of the system and is comb ined with a thresholding function. 
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Transform coding using coefficient prediction techniques 

Claims of EP0422404 

representation of an input image from the transform coefficients comprising the steps of. 

received transform coefficients, wherein said selected coefficients compnse AC coefficients. 
t'ZSgSESESSESXZZ £ gSblocK to corresponding .0 said thresholded 

additions of step d) for all of said blocks of said image received. 

2.The method of claim 1 wherein said received transform coefficients for all of said blocks have been 
quantized and comprising the further steps of: ^ nMinn - an H 

iTde^iSn^^^ 

SCS3r£S!r» s^S^pSSverse — «» op.re.ion. 

SS^^vSZ^Z^*- o, said biocKs before the generaUng of predicted 
vaSci selected ?ransform coefficients from the values of said recetved transform coefficients. 

6 The method of claim 1 wherein said received transform coefficients for all of said blocks have been 
received transform coefficients. 

7. The method of one of claims 1 to 6 incorporated in the decoder feedback loop of a transform based 
motion video encoder. 

rpnrpspntation of an input image from the transform coefficients comprising: 

SeneratogScied for selected transform coefficients in a given block from the vahjes of said 

uHoXmSns ^tcteXlaSand third means, for adding co< t es^ ,^^>™ 

^^l^^T^^ZX^ - Averse transform operation to .heresulrs 
K addfcns^erformed by said fourth means on all of said blocks of said image received, to recover 
said image representation. 
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9 The apparatus of claim 8 wherein said received transform coefficients for all of said blocks have been 

S?^ a ^S5SS3 toUw second means, for quantizing said predicted coefficient values 

S^SSlo said fourth means, for dequantizing the results of the additions before said 
sixth means applies said inverse transform operation. 

I&S^SSSIZ -t^-ESZU"-* ,0, ft. sc* g o, said seventh n,eans 
after the application of said inverse transform operation. 

1 1 The apparatus of claim 8 wherein said received transform coefficients for all of said blocks have been 
n,. a nti?ed and further comprising seventh means, connected to said first means for dequantizing said 
S^cSSiSS^^of sa?d b.ocks before said second means f^^f^ 
selected transform coefficients from the values of said received transform coefficients. 

1? Tho aooaratus of one of claims 8 to 1 1 further comprising a transform based motion video encoder 
hSU % S^^aS^hewin said first through sixth means are incorporated in sa,d encoder ,n said 

feedback loop. 

M In a oroaressive transmission scheme for still frame images wherein input images are subdivided into 
Mocts of SS said Socks of pixels are changed into blocks of coefficients by transform coding 

Presentation of an input image from the coded transmission comprising the steps of 
7^TL lrSge quantized coefficient bit values for all of said blocks of sa,d image; 
b generating t quantized predicted value for certain coefficients in a given block; 

^^X^^^X^^ corresponding quantized .efficient bit 
e? repealing eaTof the foregoing steps for all of the blocks of said image received; 

^recovering said image representation by applying an inverse transform operation to the results of the 
additions of step d) for all of said blocks of said image received. 

14 The method of claim 1 3 further compromising the steps of 

g) additionally receiving said less significant bit values from said successive stages for all of sa.d blocks of 
tSSSSR said less signfficant bit va.ues from said successive stages for all of said b.ocks of said image 

in said addition step d). 
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